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ixdividuai.lv prhscrii;!:d instruction program 

(MatJicL.atics) 

k I. INTRODUCTION 

A. Needs and Rationale 

From among the new instructional -methodologies emerging 

from research in education, the Cleveland Public Schools selected 

the Individually Prescribed Instruction (IP1) approach for cxperi- 
< — — ^ 

mental trials in teaching mathematics in two Title I elementary 

schools. Implementation of tl*c program began in the fall of 1969 

and has continued through the 1971-72 school year. This report 

is the evaluation of the tnird year of program operation. 

The IPI technique is being installed in school systems 

across the nation byjlfesearch for Hotter Schools (RBS), a federally 

funded regional education laboratory. IPI is designed -to provide 

a program of study in math that is individually tailored for each 

student on each day. The progran has six elements that distinguish 

it from conventional school procedures. They are: 

. Detailed specifications of educational objectives. 
. Organisation of methods and materials to attain 

these objectives. 
. Careful determination of each pupil 1 s present 

competence in a given subject. 
. Individual daily evaluation and guidance of each 

pupil 

. Provision of frequent monitoring of student per- 
formance m oruer to inform born the pupil and 
the teacher of progress toward an objective. 

. Continual evaluation and strengthening of the 
curriculum and instructional procedures. 

\ 1. Objectives 

Generally, this program seeks to implement a new 
instructional process in mathematics for Grades 1-6. The 
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objectives f:*31 into tv;o broad categories: (3) process 
objectives rejated to the implementation of the program in 
the target schools, and (2) product objectives related to 
chc expected results of the successful implementation of the 
program • 

a. Process Objectives 

(1) To instruct target school staff during t!ic summer of 
1972 -in individualizing reading and spelling instruction 
by means of a training program in Individually Pre- 
scribed Instruction designed and conducted by Research 
for Better Schools* 

(2) To provide an individualized progran of mathematics 
study for each pupil enrolled at the target schools by 
-implementation of the Individually Prescribed Instruction 
Program according to the format specified by Research 
for Better Schools. 

b. Product Objective 

(1) To increase the level of achievement of pupils in 

mathematics significantly beyond the level demonstrated 
by pupils in two Control schools as measured by the 
prc-post administration of standardized mathematics 
achievement tests. 

Historical Background 

The IPI system was developed by the Learning Research 

and Development* Center of the University of Pittsburg, and is 

being field tested in school systems across the country by 

Research for Better Schools. The introduction of the IPI program 

into the Cleveland Public Schools grcv: out of visits by repre- 

sentatives of the Department of Mathematics and the Division of 

Research and Development to other sciiool systems where IPI pro- 

grams were in operation. Training of the principals of the 

target schools took place in April, 1969 and of the teaching 

staff in June and July, 1969. Implementation of the program in 

the Cleveland Public Schools began in September, 1969. 



The evaluation of Years 1 and 2 of projran operation 
resulted in the following findings: 

1. Al] staff training was conducted as proposed and according 
to the fornat established by UBS. 

2. A tean of neutral observers rated the program operation as 
wcll-orp anizod and implemented. Suggestions v:crc made for 

a greater quantity and variety of Manipulative tutorials for 
use in instruction and for greater variety in dov:ntine 
assignments. 

3. An independent evaluation conducted by HBS in Year 1 
indicated that both schools showed a high decree of ac- 
curacy in correctly following tbo IPI procedures. 

4. Comparison of the performance on standardized achievement 
tests showed the fol lowing: 

In Year 1 the children in IPI schools pcrformcdsi^nificantly 
better than children ir; Control schools on one out of three 
subtests at Grade f>. At Grades 1 and 3 the children in 
Control schools performed better than children in IPI schools 
to a degree that reached or approached statistical significance 
on all arithmetic subtests 

In Year 2 the childrcinrt IPI schools performed significantly 
better than children in-Controi schools on the one standardized 
arithmetic test acini ni stored at Grade 1. At Grade 3 children 
in Control schools scored significantly higher on one of the 
two. tests administered. At Grade 5 Control" children scored 
significantly higher on one out of three tests adnini stored. 

5. Longitudinal analyses comparing children who had been in the 
IPI or Control schools for the two full years of operation 
showed only one significant difference in nath performance on 
standardized tests. Pifth graders who had been in Control 
schools for two years scored significantly higher than IPI 
children on the Computations subtest. 

6. Cross sectional analyses of changes in performance at given 
grade levels across time indicated that the IPI Program had 
little effect on performance levels over two years of 
operation. 
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. S u;anary of Prof ran Operations in Year 5 
1, Participants 

In Year 3 the IPI Program operated in the san>e ttco 
el ementa ry schools as in Years 1 and 2, Parkwood and Stephen 
E # Howe. All students in Grades 1-6 received the services 
of the pro^ran. Parkwood served 437 pupils and. Howe served 
446 for a total of 885 children rec*ivin> project services. 

During the first year of program operation (1969-70) 

approximately $137,210 was spent on the TPI Program in addition 

to General Fund expenditures in the two schools served. By 

Year 3 IPI expenditures had increased to approximate ly $185,641. 

Per nupil expenditure data are presented in Tabic 1. 

Table 1 

Per Pupil Expenditures fron IPI and General Funds 
in Year 1, Vuar 2 and Year 3 



Per Pnj«il Expenditure 



J'roj'.rari Year 


11*1 


Gcncrnl 
Fund* 


Total 


Year 1 


SI 21 


$52* 


$173 


Year 2 


$162 


$56 


$213 


Year 3 


$210 


$64 


$27-! 



*Ccneral Fund expenditures for itath instruction arc based 
on an estinatcd qf the total General Fund per pupil 
expenditure. 

The table shows that over the course of three years the per 
pupil expenditure fron IPI funds increased 74% , general fund 
expenditures increased by 23 % and the total per pupil ex- 
penditures in the project schools increased by 58^. In Year 
3 the addition of IPI funds to the general funds nornally spent 
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on natii instruction, increased the per pupil expend j ture-in 
the project Sv!«ools by 22? %. 
I). Questions to be Ansvcrcii by Evaluation 

The evaluation of Year 3 of the 1PI Progran was ad- 
dressed to four questions based on the object i\'cs of the 
projjraw: 

1. Did the project staff conplctc the training program for IPI 
(Process Objective 1) 

2 # iiov: t;crc the 1P1 schools organized for iisplcncntation of the 
4 prof-ran? (Process Objective 2) 

3. Kas the product objective related "to changes in achicvenent 
level attained? (Product Objective 1) 

* ' r 

4 # Khat has been the inpact on achicvenent levels in r?th over 
tr<c three years of operation of the nrorran? 
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HIGHLIGHTS OF FINDINGS 

r~' — : — ; — ' 

A. Surgmry of Findings 

1. Evaluation Question 1: Did the Project Staff Complete the 
Training Program for lPlT 

FINDING: Staff training in IPI Reading and Spelling did not 
take place because of a decision not to implement 
an IPI program in these: areas. Staff new to the 
IFI schools received training IPI Math. The process 
objective related to staff training was considered 
achieved, 

2. Evaluation Question 2: Hoy ffcrc the IPI Schools Organized 
for Implementation of the Program? 

» _ 
FINDING: The organization and procedures in the IPI schools 

were found to be essentially the sa:nc-as~in previous 
years when a detailed analysis shotted that the IPI 
approach was being implemented according to 
specification. The process objective related to 
^implementation of program procedures was considered 
achieved. 

a. All staff and materials for the program were provided as 
planned, although Stephen Iiov;e school reported difficulty 
in getting orders for IPI materials correctly filled. 
Materials for the program were maintained in a centrally 
located watcrials. center as proposed. 



b. In both schools the math classes v;crc scheduled and 
organized to permit the presence of the homeroom teacher, 
a consultant teacher and two aides. Some classes had the* 
services of two consultant teachers. 

c. In both schools weekly planning meetings were held among 
teachers from the sane or adjacent grade levels. 

d„ In both schools prescription writing was done during class 
time, bell time, or during free periods. Prescriptions 
were written by both the homeroom teachers and 'the 
consultant teachers. The assignment and scheduling of 
staff was such that prescriptions were always ready by the 
beginning of the math period. 

e. The staff in both schools used little of the budgeted after- 
school in-service time, but used planning meetings and 
rcgulai* faculty meetings to accomplish the same goals. 
In Stephen Howe part of the in-service time was used to 
construct manuals of remedial or enrichment assignments 
for children to worl; on during the time their. IPI work- 
sheets were being checked. 

/ 

/ 



f # itoth school s nade efforts to educate parents in *he 
operation of the IPI Program through PTA meetings, 
demonstrations, brochures and the use of a 'j^iiquc 
report card* „ V, 

3. Evaluation Question 3: Was the Product Objective fcclatcd to 

Changes in Achievement Levels Attained? 
< : 

FINDING: The achievement objective of IPI superiority over 

Control schools in pcrfomance on standardized nath 
achievement tests vras attained at one of the three 
grade levels tested. 

a^ Product Objective: IPI Superiority in Achievcnent in Math 

A sunrary cf the results of the Stanford Achievcnent 
Tests administered to IPI and Control children in 
Grades 2, 4 and 6 appears in Table A. 

Table A 

Sui2nary of Results of Achievcnent Testing in Year 3 





SUPERIOR GROUP ON l-ACJi MATH 


SHSTKST 


Grade 


Arithmetic 
Computations 


Arithmetic 
Concepts 


Arithmetic 
Applications 


'. 2 , 


None 


None 




A 


Control 


Control 


IPI 


6 


Control 


IPT 


IPI 



IPI chijdren scored higher than Control' children on one 
out of three subtests at Grade A, two out of three 
subtests at Grade 6 and none out of ttco subtests at Grade 
2. The product objective is considered achieved at Grade 
6, The performance in Year 3 is soncv:hat better than in the 
previous year vhen IPI children scored higher on only one 
out of one subtests administered at~onc of the three grade 
levels tested. 

b. Progress in the IPI Continuum 

A pre and post analysis of the percentage of children- 
working at various 'skill levels in the IPI continuum 
. showed that children at different grade levels progressed 
throug^i the continuum at approximately the sane rate. 

4. Evaluation Question 4: Khat has been the Inpnct on Achievcnent 
Levels in Hath over the Three Years of Operation of the Program? 

FINDING: The results of longitudinal analyses provide no 

• evidence of IPI superiority innath achievement over 
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three years of operation. Cross sectionai analyses 
provided sonc evidence of a gradual rise in achieve- 
ment levels at certain grade levels. 

a. Longitudinal Analysis 

Samples of children t:cre selected Kho had been 
in the IPT and Control schools for the entire three years 
of operation of the program. These children were compared 
for deferences -in achievement levels at tiic end of the 
three years". Three sanplcs of children v;crc tested: 

# Grade 1-2 sample - Children entered program 
in Grade 1, Year 2 and were compared in 
s Grade 2, Year 3. 

* 

. Grade 2-5-4 sample - Children entered pronran; 
in Grade 2, Year 1 and were conparcd in Grade 
4, Year 3. 

. Grade 4-5-6 sample - Children entered program 
in Grade 4, Year 1 and were conparcd in Grade 
6, Year 3. 

Table !$ shows a sundry of the results of the achievement 
comparisons madej^n-'cen IPI and Control children in the 
longitudinal samples. 

Table 15 

Summary of Results of Longitudinal Achievement Comparisons 
Between IPI and Control Children 





SUPl:!U0PwGHOl5i> OM HACH MATH .SUBTEST 


Longitudinal 
Sample 


Arithmetic 
Commutations 


Arithmetic 
Concepts 


.Arithmetic 
Ann li cat ions 


Gr. 1-2 


None 


None 




Gr. 2-3-4 


Control 


None 


None 


Gr. 4-5-6 


Control 


Control 


Sonc 



Of the three 'ongitudinal samples tested, none showed 
any significant differences in favor of IPI, These 
results arc similar- to those in the previous year of 
operation. 

b. Cross Sectional Analysis 

Analysis of the changes in performance levels across three 
years at given grade levels in IPI schools revealed the 
following results: 
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(!) Of tho three grade levels examined (firades 3, 5 and 
6) only at Grade 6 was there any evidence imlicat-inc 
a possible salutary effect of the IPJ Prop-ran on 1 
achievement .levels . At p rade 6, performance levels - — 
on the Concepts and Applications subtests appeared 
to he gradually risino in I PI schools in the face 
of generally declining performance in other schools* 
In Grade 3, changes in performance in IP1 schools 
were generally paralleled by similar ciianf.es in 
other schools, and in Grade S the pcrfornancc in 
IPI schools declined relative to other schools, 

B. Implications- and KcomHcndatioris 
1-i Discussion of Results 

The results of the third year of operation of the 
IPI Program have shown that (as far as can be determined 
— without a detailed analysis of specific diagnostic and pre- 
scription writing procedures) the IPX Progrr.iv continues to be 
implemented according to plan. The question cLst remains is 
the effectiveness of „the program in raising the math achievement 
levels of the children served. The prc-post analysis of 
progress through the skill levels within the IPI cent j mum 
showed that the children arc moving along the continuing but 
because of the lack of norms against which to compare their 
progress, it is difficult to determine whether they arc pro- 
grossing at a "normal 11 rate. The cvaluator must, therefore, 
rely on information from standardized achievement tests to 
draw conclusions about the impact of the program on math 
performance. 

The achievement data from the third year of operation 
showed that the children in 11*1 schools fared somewhat better 
in comparisons with children in Control schools than they did 
in previous years. The results, however, did not indicate a 
strong impact, except at Grade 6. Similarly, cross sectional 
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analyses of changes in performance levels at given grade 
levels across the three ye;jrs of operation showed little 
evidence of improvement except at Grade 6. Longitudinal 
analyses of the performance o£ only those children who had 
been in the program-contimiously r for two or three years showed 
no evidence at all of superior performance by IPI children. 

As discussed in the evaluation report for Year 2 of 
operation, there arc several possible reasons why a greater 
impact on achievement has not been evident. The first 
possibility is, of course, that the 11*1 .approach simply does 
-not work any better than the traditional methods of math 
instruction. Other possibilities exist, however. It is 
possible that the standardized achievement results do not 
adequately ncasurc what is taught in the IPI continuum, (a 
possibility that raises questions as to what is taught in 
the IPI continuum). One way of investigating this thesis is 
to analyze the standardized tests item by item to determine 
whether the questions reflect the skills and concepts covered 
in IPI. The team of consultant teachers in Stephen Howe 
school was assigned this task, and their analysis of the 
arithmetic subtests of the Stanford Primary I, Stanford Pririary 
II, Stanford Intermediate I and Stanford* Intermediate II 
achievement tests showed that an average of 94% of the items 
on these tests were covered in the IPI continuum. The range 
of correspondence between the Stanford tests and the IPI 
continuum was from S2% to 100% for nine arithmetic subtests. 

It would appear that the standardized tests do measure the 

/ 

sane things that arc taught in the IPI continuum. 
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It is possible, of course, even though the standardized 
tests measure basically the sane skills as arc taupht in IPI, 
that the language of the tests is sufficiently different fron 
the language used in IPI to impair performance on the tests. 
For example, one o . the Stanford Intermediate I 

Arithmetic Concepts makes use of the terms multiplier , multi- 
plicand and minuend , but these terms arc not used at all in IPI. 
To investigate this potential problem, the consultant teachers 
in the IPI schools constructed achievement tests directly from 
the IPI materials. Tests were constructed for Grades 3 and 5, 
and items were selected that, in the judgment of the IPI staff, 
reflected the instructional activities in which all or most of 
the pupils had participated. Since the test items reflected 
what most of the children had hcen taught in the IPI continuum, 
a mastery level performance was expected. It was anticipated 
that each item would be answered correctly by at least 75% 
of the children. The results showed, however, that no more 
than 36% of the items on any of these tests were correctly 
answered by 75% or more of the children. In fact no more than 
57% of the items on any of the tests were correctly answered 
by more tlian 60% of the children. These results are not con- 
clusive, because these IPI tests are as yet in unrefined form, 
but they do suggest that lack of evidence of improvement in 
math achievement may not be due to the use of standardized 
tests. 

Another possibility for explaining the apparent lack 
of progress in raising achievement levels is that the IPI 
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system teaches the math skills that the Cleveland Schools wants 
taught, but that it teaches them in a different sequence from 
that normally followed. That is, by the time he leaves 
elementary school, a child in IPI may have covered the same 
concepts and skills as a child not in IPI (hopefully with 
greater mastery) , but at any given point in time during their 
elementary school careers, they may be working on entirely 
different skills. If this is the case, the only valid assess- 
ment of IPI possible is a longitudinal study in which achieve- 
ment levels of IPI and Control children are compared only after 
the IPI children have spent all or most of their elementary 
school years working in the IPI system. One problem with this 
approach 9 however, is that the percentage of children Remaining 
-in the schools that long is relatively small. Less than one third 
of the children present in the IPI schools during Year 1 were 
still in_Jhe schools by the end of Year 3. Therefore, the 
whole question of whether the IPI Program might have an impact 
on achievement if children participated for their entire 
elementary school tenure may be academic. 

Finally, despite the indications that the program 
guidelines have been faithfully followed by the IPI staff, over 
the course of three years it is possible that deviations from 
the precise testing and prescription criteria established by 

the program's designers have crept into the operation. If so, 
« 

these deviations may be reflected in lack of improvement in . 

math achievement. It was recommended in the evaluation report 

j 
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for Year 2 that another evaluation of operations accuracy be 
conducted by Research for Better Schools, but that organization 
has reported that they no longer perform that service. Self- 
monitoring instruments have been developed, however, and their 
use in the Cleveland IPI schools may yield valuable information 
as to the accuracy with which the IPI procedures arc being 
followed. 

In the long run, the decision about thefuture of the 
IPI Program will have to ^balance program costs against program 
benefits. There is some evidence that achievement levels in 
math in the IPI schools may be improving slightly at certain 
grade le ^ls. But is is also clear that over the course of 
three years the cost of the program has risen 74% to $210 per ' , 
pupil. The question becomes whether the present or future 
gains in achievement arc great enough to justify the expense, 
or whether more efficient and less costly avenues to the same 
ends can be found. 

At any rate, the results of field tests of experimental 
programs in the schools must be interpreted in light of the 
number of influences over which the evaluator lias no control; 
experimental treatment populations change, control populations 
change, samples may become biased through non-random attrition. 
All of these factors may cause changes in performance that have 
nothing to do with the experimental treatment und?r investiga- 
tion. Evaluation must, therefore, take a long and cautious View 
of results 'and data must be examined over a period of several' years 
before administrators can state with some degree of confidence 
that a given program is or is not effective. 
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2. RccofisnendAtions 



a. It is recommended tiiat the IP] Program he i.o* tinned in 
the sane two schools in which it presently operates for 
the next one or tv:o years until sufficient oat a arc 
available for an accurate appraisal of lonr terra program 
effects, 

b. It is rcconmcndcd that a decision on expansion of the 

IPI Program to other schools be deferred until the results 
of a lonj* tern evaluation are available. 

c. It is recommended that the development and refinement of 
achievement tests constructed from the IPI materials 
proceed. ■ * 

d. It is recommended that self-monitoring procedures be r 
instituted in each IPI school, using the instruments 
developed for this pfirposc by RBS. > 
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PROJECT DKSCIMPTION' 

^* l } ^^t:ici pan 1 Character! sti es 

Students in both Parkwood and Stephen I:. Howe schools 
have exhibited a pattern of increasing deficit in achievement • 
in math as they progress from Grade 1 through Grade 6. Before 
the IPX Program was begun, the average achievement level in 
math was over one year below grade level norms by the end of 
the sixth grade. Both schools arc "poverty schools" with an 
average of 42% of the children on public assistance. Pupil 
mobility rates arc high, averaging 70% during the school year 
1971-72. Table 2 shows the number of students served in the 
project by grade level. (Sec Appendix A for a breakdown by 
school) . 

Table 2 

Number of Pupils Served by IPI Program by Grade Level 

Grade Level Number of Children 

EMR - ' 18 . 

1 - 150 

2 - 168 

3 - 147 
4-127 

5 - 117 

6 - 156 

Total 383 

B. Project Operations (Program Plan) 

1. Organizational Details and Activities 
The basic plan for IPI is as follows: 

a. The subject area of mathematics is structured into a 
hierarchical continuum of several hundred learning 
objectives along which the student progresses at his 
own pace. The objectives arc sequenced so that before 
a student begins to study a new mathematical process, 
he has already mastered those skills that are pre- 
requisite to learning the new process. Thirteen areas 
of mathematics arc covered: 
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Numeration 
Place Value 
Addition 
.Subtraction • 
Multiplication 
Division 

Combination of Processes 

Fractions 

Money 

Tine 

Systems of Measurement 

Geometry 

Special Topics 

• Fach of these areas is divided into ski] 1 'levels A through 
G, each skill level involving progressively more complex 
processes. 

b. Placement tests arc given when a child enters the program to 
determine his strengths and weaknesses and to detcrnine at 
what level lie will begin working in each area of math. At 
the beginning of each unit a prc-tcst is given to identify 
weaknesses in that unit, and then the child begins work in 
the Standard Teaching Sequence (STS) leading bin through the 
pre-detcrmined sequence of objectives. Curriculun Knbedded 
Tests (CP/is) are given as part of the STS and serve to test 
skill Mastery within a unit. At the completion of a unit, 

a post test is given to confirm nast cry of the entire unit. 

The teacher, on the basis of the test information, writes 
learning prescriptions for each child, detail inp his work 
assignments. These nay most often- include assignments in 
the STS but may also include teacher constructed worksheets, 
textbook assignments, manipulatives, etc. All learning 
. materials arc kept in a centrally located material center 
staffed by aides. 

d. A folder is maintained on each cliild containing his 

prescriptions, current work assignment and ids record of 
progress. At the beginning of each class, the child takes 
his folder, reads his prescription for the day, and proceeds 
to the materials center to get the numbered worksheets 
dealing with his objectives for the day. Or, he may continue 
working on the previous day's prescription. Most assignments 
consist of the child working alone on his individual assign- 
ment, but the teacher nay also prescribe individual tutoring, 
small group work or large group work. 

c. As a pupil completes sections of his work assignment lie, 
turns them in to a teacher aide who checks and returns 
them. The period during which a pupil's worksheets are 
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being checked is called "downt jiae". After his u-orl: 
has been checked, the pup j J either asks F or a new 
prescription from The teachcj or asks the teacher 
for assistance. Pupils request new prescriptions or 
assistance from the teacher by displaying a red 
flap, on their desk. 

f. Tiie maintenance of the materials center and all 
checking of papers are the province of aides, 
freeing the teacher for teaching, evaluation, diagnosis, 
and prescription writing. Floating teachers without 
horccroon responsibilities arc provided to increase the 
number of professional staff working directly" with the 
children. 

2. Staffing 

The program plans called for the addition of several 
staff to each project school in order to implement the IPI - 
procedures. Included were ten consultant teachers whose 
responsibilities were to work with the homeroom teachers in 
preparing prescriptions and in diagnosing and evaluating 
student progress. Consultant teachers were also to be re- 
sponsible for supervising the teacher aides, kightccn teacher 
aides were proposed to handle clerical work involved in cor- 
recting students' worksheets, to maintain the I!>1 materials 
center, and to maintain project records and individual student 
files. 

3. In-Service Training 

Program plans provided 30 hours of in-service train- 
ing in the IPI math technique for teachers and aides new to 
the project schools. In addition, the entire staff of one IPI 
school was to receive 50 hours of training during the summer 
of 1972 for implementation of an IPI Reading and Spelling 
program. - 
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Six after school inscrvice neetings for ll'l staff 
were planned for review of curriculurt ciianp.es planned by Rl'-S 
and for the development of IPI-relatcd instructional materials. 
A . Parent Involvement 

Because of the emphasis in J PI upon the nastcry of 
behavioral objectives v:ithiu a rath continuum rather than the 
traditional narking system, comaunication with the parents about 
the program was considered important by program planners. Plans 
for increasing parent understanding of the program included 
parent meetings, classroom demonstrations and displays', a unique 
report card system, and packets of literature describing the 
program . 
EVALUATION 

Evaluation activities during Year 3 focused primarily on assess- 
ment (>f the product objective related to achievement levels. Although 
data related to success in achieving the process objectives were col- 
lected, the smooth iraplcncntatioii of procedures during Years 1 and 2 
suggested that less emphasis on assessment of implementation was 
required. 
A, Basic Design 

The evaluation design employed cither a status description 
or a cross-nested experimental-control model, depending on the 
nature of the objectives under considers ^on. The status des- 
cription was built on the results of interviews with project 
* « 

personnel and the examination of school and project records. Tor 
expcriricntnl-control comparisons, two Control schools (Hough and 
. Hazclctall) were selected to match the VH schools as closely as 



possible on percentage of students on public assistance, at- 
tendance, nobility, achicvcncnt levels, intelligence levels, and 
racial composition. These Control schools received the services 
of other Title I and DPPF projects. Appendix B lists the 
characteristics on which the IPI and Control schools were original- 
Jy rsatched and shows the degree of patching attained. 
B. Presentation of Findings 

Tlic evalu;^t-5?i ; i?%a;occdu'rcs were address c<t=JLo four basic 
questions about project oj"»mtj«*sis and results, and the findings 
will be organized in the sane way. Bach basic evaluation question 
will be pospd, followed by an examination of the data collected on 
the objectives that arc pertinent to that evaluation question. 
Included where necessary will be a description of "instrumentation 
and data collection procedures. 



E VALUATION QUES TION" 1 
DID Till: PROJECT STAFF COMPLETE HIE TRAIN" I. NG PROGRAM FOR IPI? 

1 . Data Col lection 

Data related to Evaluation Question 1 were collected 
through interviews with the principals of the project schools. 

2 , Process Objective 1: Staff Training in IPI Reading and 
Spcl ling 

The original program plans for Year 3 included train- 
ing the staf* 7 of one of the IPI _ schools in IPC techniques for 
reading and spelling with the aim of implementing a reading 
and spelling program in the fall of 1972. However, these 
plans were postponed after the evaluation of Year 2 indicated 
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that a more coup] etc investigation of the IPI path profrar. 
would be required for a decision on the effectiveness of the 
IPI approach in general. Therefore the training in IPI Reading 
and Spelling did not take place. 
Staff Retraining in IPI Math 

.Staff turnover and additions required the training 
of several new teachers and aides in the IPI technique. In 
both schools, five full-day training sessions for new staff 
were conducted during the v:cck before school. In Stephen Howe 
school, where both teachers. and aides were in need of "training, 
the sessions were conducted by the consultant teachers. In 
Parkwood, where only one aide was in need of training* the 
sessions were conducted by the head teacher aide. 

The teachers were trained using a teacher-training 
curriculum devised by RBS. Materials included behavioral 
objectives, self instructional Materials, and recommended learn 
ing settings. Training activities included concept building 
related to IPI, an, analysis and application of the concepts to 
iPI, practice in using. IPI skills and materials as routine 
exercises, and discussion designed to provide an opportunity 
for clarification and expression of reaction to IPI. The 
training sessions were designed to equip the teacher to 
conceptualize a model of IPI as a basis for instructional 
decision making and to plan and conduct IPI in the classroom. 

Process Objective 1 is considered achieved. 
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KVAi.ii/vnoN' 'ji'hSTin:; 2 

110K bhKh Till: IPI SCHOOLS 0KGAMZKI1 POR 
IMPLhMl-NTATKW OF Tl.K PROGRAM? 



Data Co] lection 

Data related to Evaluation Question ? were collected* 

through interviews with the principals of the project schools. 

Process Objective 2: To Provide an Individualized Program o r 
Math Instruction t^rou^u Implementation of I! 1 ! Procedures , 

a. . Ma terials 

Both IPI schools mairtaincd a materials center 
where testing and prescription materials required by the 
/ prograw were stocl.ed. The materials centers v:ere under 
^ tiic continuous supervision of a teacher aide. At Stephen 
Howe considerable difficulty was reported in keening the 
materials center stoc) ed with the needed IPI materials. 
The difficulty lay i 1 orders for natcrials being slowly 
and incorrectly filled by UtfS. TJic problem was compounded 
by changes in the curriculum materials instituted by UBS. 
These changes rcruircd wholesale replacement of large 
vol limns of STS Look lets. 

b. Staff 

Fivt; consultant teachers without homeroom as- 

s - - — 

signmcuts v:e;c assigned to each school, an increase of 
one per school over Year 2. These teachers worked ex- 
clusively lji the area of IPI, rotating auong classrooms 
to help tie hoiitcroom teachers in evaluation, diagnosis and 
prescription writing during the IPI math periods. 
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Both schools were also assigned teacher aides for 
work exclusively in the IPl Program. Parkwood received the 
services of eight aides and Stephen Howe was assigned ten. 
The aides were responsible for all record keeping, distribution 
and maintenance of materials and supplies, the operation of 
the materials center, and scoring the students 1 worksheets. 
Organisation and Scheduling 

In botii schools the lack of rcadinf skills hindered 
trie inplencut ation of the program Kitii children in Grade 1. 
In Stephen Howe an intensive reading vocabulary .program geared 
specifically toward the language used in the IPI materials, was 
held during the first week of school. This vocabulary program 
made it possible for Grade 1 children to begin work in the IPl 
^coritjnuun in September. In Parkwood, Grade 1 children began 
working in IPI in January. 

In both schools the scheduling was arranged so that 
each math class had* the services of the homeroom teacher, a 
consultant teacher and two aides. Jn both schools sonc classes 
had the services of two consultant teachers. In Stephen Howe 
two consultant teachers worked in each classroom in Grades 1 and 
2. In Parkwood two consultant teachers worked in the largest 
classes, irrespective of grade level. 
Planning 

In both schools IPI meetings among homeroom teachers 
were held regularly once a week to discuss problems, share 
ideas, plan downtime work, review progress, and select children 
with similar problems for small group instruction. In both 
schools the ncctings involved teachers frori the same or 
adjacent grade levels. 



In both schools the planning recti ng day for the 
teachers was "ser.inar day" for the children. Halo the hone- 
room teachers net,, their classes were covered by the consultant 
teachers who conducted scninars to introduce new IPI concepts 
to the children. % 
Prescription Writing - 

At both schools prescriptions for childrcns' work 
were written by honeroon and consultant teachers during class 
tine, free periods and during bell tine. There v;as no special 
tine set aside specifically for writing prescriptions. If 
prescription writing v;as not completed by the end of the math 
period, the standard practice in both schools was for a con- 
sultant teacher to finish it later in the day to insure that 
all prescriptions were ready in tine for the math period the 
next day. This practice was made possible by the addition of 
one extra consultant teacher in each school in Year 3. 
Jn-Service Meetings 

Aside from the in-service training for nev: teachers 
and aides that was discussed under Evaluation Question 1, 
money teas budgeted for six after school in-service Meetings in 
each school. The purpose of the meetings was to review cur- 
riculum changes planned by RBS and to develop IPI-related 
instructional materials. 

Both schools conducted just one such after-school 
meeting, but used the regularly scheduled planning and faculty 
meetings to accomplish the s:ine ends. In Stephen iiowc part of 
the in-service time was used to construct manuals of activities 



for teachers to assign students during "downtime", the period 
during v.hicli student*, waited for their worksheets to be checked* 
Four nanuals were developed, one for grades 1 and 2, one cadi 
for crudes 5 and -1 and one for [trades 5 and 6. The manuals 
were designed to provide either enrichwent or remedial assign- 
ments related to the area and skill level of the path continuun 
in wiiicii the student was working. 

g. Parent Involvcr.cnt 

Both schools continued efforts to communicate the 
nature and purpose of the IPI Program to the parents of the 
children served. These efforts included demonstrations of 
the IPI technique to PTA meetings and groups of parents visiting 
the classrooms, distribution of brochures describing the program, 
and the use of a report card that shows where the child placed 
in the math continuum and explains his progress. 

h. Sunnary of Eva 1 is:! t ion n uestion 2: How Kere the IPI Schools 
Organized for lr.pl encntntion o r the Pro';rap? 

The results of the first two years of operation of 
the IPI Program showed that both schools were accurately fol- 
lowing the IPI procedures specified by UBS. This conclusion 
was derived fron both local evaluation data and the results of 
a process evaluation conducted by UBS. fiiven these results 
during the first two years, it was not considered necessary to 
examine the process aspects of the program in such depth during 
the third year. Since the data that were collected during the 
third year indicate that essentially the sane organization was 
maintained as during the first two years, it seens safe to 
assume that implementation of the IPI Program continued to 
accurately reflect the procedures specified by UBS. Process 
Objective 2 is considered achieved. 
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i-VAUiATior: quustton 3 

WAS Till: PRODUCT OBJECTIVE KKLATI1D TO CHANGES 
IN ACHIEVEMENT IJSVHLS ATTAINED? 

1 . Product Objective: 1PI Superiority over Control Schools in 
Math Achievement 

a. Data Collection 

In order, to assess the impact of the- IPJ Program on 

achievement levels in basic math skills, conparisons were made 

between IPI and Control schools on performance on the arithmetic 

subtests of the Stanford Achievement Tests. The tests were 

administered to a sample of children from Grades 2, 4 and 6 

in IPI and Control schools September, 1 *)71 and again in 

May, 1972. 

Data were collected from students in regular classes 
only. Students in any kind of special class such as Major 
Work, Enrichment , Listening Post or EMI wore not tested. " 
Table 3 shows the levc* and form of the test administered to 
each grade level and the number of students included in the 
analysis of the data. 

Table 3 

Form and Level of Stanford Achievement Test Administered 
to Grades 2, 4 and 6 in IPI and Control Schools, 
and Size of Sample Tested 



tfradc 
Level 


Sampl 
IPI 


c Size 
Control 


Level and Porn of Test 


2 


108 


111 


Pre - Primary I, Form X 
Post - Primary II, Form W 


4 


97 


121 


Pre - Intermediate I, Form X 
Post - Intermediate I, Form W 


6 


125 


210 


Pre - Intermediate II, Form X 
Post - Intermediate II, Form W 



Data analysis was performed" on the scores of only 
those children for whom both pre and post test "scores were 
obtained. Consequently, the results reflect the achievement 
of only those students who attended IPI or Cditrol schools 
for the nine-month period from September, 1971 to May, 1972. - 

Data analysis was performed on the raw scores 
attained on the tests, i.e., the number of correct responses. 
Raw scores on the Stanford Achievement Tests arc often trans- 
formed into grade equivalent scores to show a studei i s 
performance relative to the national norms. Grade equivalents 
arc very useful for descriptive purposes, but because they 
are _not nil equal-interval scale, raw scores were preferred 
for statistical analysis. 

IPI and Control schools were compared with respect 
to post-test math achievement scores. To avoid, the problem 
of the post score differences bcinf mere Ij^^jfcf lection of 
initial differences in pre scores, the data v;erc~analyzcd 
by means of multivariate analysis of covariance (MANCOVA) • 
MAKCOVA determines the effects that selected variables 
(covariatcs) nay have on post scores and adjusts the data 
for these effects before making comparisons. In the present 
analysis the students* pre- test math achievement scores and 
their PLR scores were used as covariatcs, and the IPI and 
Control schools were compared on post-test performance after 
adjusting for the 'effects of these covariatcs. 

h 
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l>, Aclii ^v orient R esults -I'M vs. Control 

Table 3 shows the results of comparisons between 
the post-test math performance of IPI and Control children 
in Grades 2, 4 and 6. In tiie analysis, math prc-test scores 
and PLR scores were used as covariates. Prc-tcst scores 
were adjusted for unequal N, and post scores were adjusted 
for unequal N and the effects of the covariates. Pull 
statistical data appear in Appendix C. 
• Table 3 

Mean Adjusted Post Arithmetic Subtest Scores for Children 
in Grades 2, 4 and 6 in IPI and Control Schools 



.MKAN AiUi'STKI) ARlTi?'>l;TTC € 5Hlfl"l-ST SCOPKS 



Grade Group 


Computations ' Concents 


Applications 


Post Superior 
Score Group 


Post Superior 
^corc Group - 


Post Superior 
Score Grou"» 


2 IPI 

Control 


21 • 2 None 
21.8 


\t:l *- 




A IPI 

Control 


12 * 8 Control" 
16.6 


13 - 4 Control*** 
14.6 


1 1 o 

TPT**** 

10.4 


6 l\\ 

Control 


lr ? Control* 
16.3 


12.3 tnj** 
11.4 


IPI* 
13.2 111 



*£<.00()1 
**n<,02 
***£<. 06 
****£ <.095 

The table shows that during Year 3 the IPI children 



scored higher than Control children on the Concepts subtest 
at Grade 6 and on the Applications subtest at Grades 4 and 6. 
The difference on tiic Applications subtest was statistically 
significant at Grade 6 and approached significance at Grade 
4*. The Cc .trol children scored significantly higher than 
the IPI children on the Computations subtest it Grades 4 and 6. 
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On the Concepts subtest Control children out scored I 1 *! 
children at Grade '1 to a degree that approached statistical 
significance. These results do not show strong and con- 
sistent differences in favor of IPI, although they arc 
somewhat inproved over the results from the previous year. 
In the previous year, significant differences in favor of 
IPI were found on only one subtest at one grade level. 
The present results nust be characterized, however, as scat- 
tered and inconclusive, despite the improvement over Year 2. 
What IPI strength there was appeared primarily at Grade 6. 

At Grade 2 a significant Sex x Treatment inter- 
action was obtained on the Computations subtest « The inter- 
action effect is illustrated in Figure 1. The interaction 
showed that tShc*lack of a significant difference between IPI 
and Control "c?3^TOcn* was due to the differential performance 
between boys and girls in the Control^schools. (Pull 
statistical data on the interaction effect are available in 
Appendix C.) 




Pig. 1 Sox x Treatment Interaction on 
Computation Subtest at Grade 2. 



In orde: to show the math pcrfornancc of the IPT 
and Control children in relation to grade level norms, the 
pre and post raw scores were transformed into grade equivalent 
scores and arc presented in graphic -form in Figure 2. The 
results for the throe grade levels are plotted on the sane 
scale so that the relationship, anong pcrfornancc and the 
norms will be comparable among grade levels. In reading the 
figure, two points should be kept in s mind: Grade equivalent 
scores are not an exact reflection of the raw score obtained 
on the test and at tines will not accurately reflect dif- 
ferences in raw scores. Further, the grade equivalent 
scores used in Figure 2 were computed from the observed 
raw score weans, not from the means used in the statistical 
analysis, which were adjusted for the effects of unequal N 
and covariatcs. 

Figure 2 shows that at Grade 2 both I PI and Control 
children were performing at or close to the norm level on 
both math tests. In Grades 4 and 6, however, the IP] child- 
rcn were functioning a year and sometimes up to two years 
below the norm with little evidence of progress toward the 
norm during the year. The exception was on the Concepts 
subtest at Grade 4 where IPI children were eight months below 
the norm at tiie prc-test and only three months below at the 
post-test. Overall, however, it is evident that there was 
a performance deficit that increased as a function of grade 
level . 
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Grade 2 



Conmitat ions 




Concepts 




IPI 



X x Control 



Pre 



Post Prc- 



Post 



Computations 




Grade 4 
Concepts 




Appj ications 




Post Pro 



Post 



>re 



Post 



u 

c 
o 



6.8 

6.4 
6.0 

5.6 
5.2 
4.8. 
4.4. 



Computations- 



Grade 6 
Concents 



Applications 




.. x * 



JL 



Pre 



Post Pre 



Post Pre 



Post 



I ; ij». 2 Mean Math Grade hquivalent Scores for IPI and- Control Children 
in Grades 2, 4 and 6 in Year 3. 
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Achievement Kcsults - i'^ys vs. Girls 

bxar.inatic:n of the nath achievement results for sex 
differences revealed significant differences at two grade 
levels. At Grade 2 hoys scored significantly higher than 
girls on both subtests wjrcinistcrcd, Computations and Concents. 
At Grade 6, boys again scored significantly higher than girls 
on two out of three subtests administered^ Computations and 
Concepts. Statistical data appear in Appendix I). 
Achievement Results - Howe vs. Parkwood 

The achievement test results were examined 
for differences between the two IPI schools, and several sig- 
nificant differences were found. At Grade 2 children in Park- 
wood scored significantly higher than children at Howe on one 
of tne two subtests administered, the Concepts sui)test. At 
Grade 4 children at iiowc scored significantly higher on the 
Computations subtest and scored higher to a degree that ap- 
proached statistical significance on the Concepts subtest. 
Tjiis rciM?«J6hts Two out of three subtests at Grade A. At 
Grade 6 children at iiowc again scored significantly higher 
on the Computations subtest, which represents one out of three 
subtests administered at that grade level. In general, per- 
forrcancc appeared somewhat better at Howe at the middle aricl~~ 
upper grade levels and better at Parkwood at the lower grade 
levels. Static *al data appear in Appendix b. 
Smnnary of Standardized Acnicvencnt Test Data 

Table 4 summarizes the nunber of nath subtests on 
which the IPI children at each grade level scored significantly 
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higher than Control children. The table shows the results 

, - > as reported in each of the three years of nrogran operation. 

Table 4 

Number* of Hath Subtcsts-on Which IPI Children 
Scored Higher Titan Control Children at Kach 
Grade Level in Kach Year of Operation 



Year 


1 


Year 




Year 


3 


Grade 


Math 
Subtests 


Grade 


Math 
Subtests 


Grade 


Math 
Subtests 


1 


o/i 


1 


1/1 


2 


0/2 


2 


0/2 


3 


0/2 


4 , 


1/3 


4 


0/3 


5 


0/3 


6 


2/3 


5 


1/3 










6 


0/3 











♦Denominators refer to the number of subtests administered, and 
numerators represent the number of subtests on which IPI children 
scored higher 

The table shows that over the three years of 
operation of the program there has been some improvement in 
the number of instances in which IPI children scored lusher 
than Control children on the math subtests administered. 
No strong pattern of superiority in IPI schools has emerged, 
however. Considering the achievement results in Year 3 in 
light of the results in JrJvc previous two years of operation, 
it may be concluded that some, although limited, progress 
has been made toward attaining the achievement objective of 
the IPI Program. 
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f ♦ A chievement in tno i'mtU Contin uum 

In order to dctcminc v.hcthcr children were making 
progress within the IPI continuum, data v;crc gathered to show 
the percentage of children working at various skill levels 
within the i3 areas of mathematics outlined by the program. 
These data were gathered fror. a sample of children at each 
of the grade levels 1 through 6 in Scptcnber, 1971 and again 
in May of 1972. For the purposes of this evaluation report, 
the data from the two schools were combined. 

Figure 3 shows the percentage of children working 
at different skill levels in September and in May of Year 
3. The figure shows that the percentage of children working 
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at the lower skill levels (ABCD) decreased from September- 
through flay, while tiiC percentage working in the higher range 
of ski 1 J levels (HFG) increased. Those data indicate progress 
fron the lower to the higher skill level range during the 
course of the year. 

Figure 3 also shows that the percentage of children 
working in the highest skill level range .(G) increased very 
little from September to May. In order to determine if the lqvT 
percentage of ciiildrcn placing at the upper skill level was 
due to lack of progress by children in the upper grade levels, 
the data were analyzed by grade level. Figure 4 shows the 
percentage of children- working -it various skill levels for 
Grades 1 and 2 combined, Grades 3 and 4 combined, and Grades 
5 and 6 combined. Again the percentages obtained in both 
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Fig. 4. Percentages of children working in IPI skill levels A through Gin 
September and May of Year 3 for Grades 1 and 2, Grades 3 and 4, and 
Grades 5 and 6. 
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September ami May are presented. The figure shows that for 
etch of the three groups of children the median of the 
distribution increased by approxinately one skill level across 
the 13 areas of mathematics from September to May, indicating 
that tlie children at all grade levels- were progressing at 
approximately the same rate. The reason that nore children 
did not place in the upper skil! level appears to be 
that, giy,cn the observed rate of progress, few children were 
within striking range of these skill levels. 

HVALliATlON OUKSTIOX 4 

WHAT HAS iM-LV Till: IMPACT 0N ! ACI 1 1 KVKMKXT LI:VI:LS 
IM MATH OVhK WW T\\\\\± Vi:AK.S UP OPhKATJOM OP T M 1: PROGRAM? 

1 . Hcncm 1 Procedures 

•* To determine the inpnet on reading and nath achieve- 

ment over the full three years of operation of the IPJ Program, 
tv:o types of analysis were conducted. The first was a longi- 
tudinal analysis in which the achicvcr.ei.f performance of a 
group of children was monitored throughout the three years of 
operation. The second analysis was cross sectional and involved 
monitoring changes in the achievement levels demonstrated at 
certain grade levels across the three years of progrsm operation. 

2. Longitudinal Analysis of Achievement 
a . Longitudinal Data Collectio n 

• The scnedule of acnicvencnt test administration 

over the three years of operation of the 1PI Program 
allowed an examination of the performance of 1PI and 
Control children on a lonp.iturtjnpl basis. Tiic number 
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of children for whon data are available over the three 
year period and the grades in which they were tested are 
presented in Table 5. (The form and level o' ? the tests 
administered at each grade in eacli year arc available 
in Appendix F.) 

Table 5 



Sources of Longitudinal Samples Over Three 
Years of Operation of the IPI Program 



Longitudinal 
Sample 


Grades Tested 


Longitudinal 


Sample Size 


Year 1 


Year 2 


Year 


3 


I fig 


Control 


Gr. 1-2 




Gr. l 


Gr. 


2 


31 


68 


Gr. 2-3-4 


Gr. 2 


Gr. 3 


Gr. 


4 


35 


62 


Gr. 4-5-6 


Gr. 4 


Gr. 5 


Gr. 


6 

... i 


72' 


127 




Table. 


5 shows 


that 


the 


schedule of 


achievement 



testing generated three longitudinal sanplcs. Two years 
of data arc available on children who began the program in 
Grade 1, Year 2 and completed Grade 2 9 Year 3 (1-2 longi- 
tudinal snnple). Three years of data arc available on 
children who began the program in Grade 2 9 Year 1 and who 
completed Grade 4, Year 3 (2-3-4 longitudinal snnple). 
Three years of data arc also available on children who 
began the progran in Grade 4, Year 1 and completed Grade 
6, Year 3 (4-5-6 longitudinal sample). 



The longitudinal analysis was performed on the 
achievement data of only those children who were enrolled 
in the I PI and Control schools during the entire longitudinal 
time frar;. Children who. entered or left the schools during 
the longitudinal tine franc were not included in the analysis. 

An important point to be remembered in interpreting 

the longitudinal analysis is that the combination of high 

pupil mobility rates and imperfect data retrieval net hods 

resulted in a decrease in the size of the longitudinal 

sample each year. Consequently, the number of children in 

the longitudinal samples after three years represented only 

about one third of the children who were originally enrolled 

in the schools. The rest of the original enrol lees had 

cither left the schools or did not have complete achievement 

data. The sample that remained represented the most stable 

elements of the pupil population in terns of mobility and 

was in that sense a biased sample. The possible effect of 

this bias on achievement results is unknown. 

Longitudinal Analysis - Results 

- -a 

Multivariate analysis of covarinncc was performed 
on the mean Year 3 post math scores of 1PI and Control 
children in the three longitudinal samples. PLR scores 
and math pre scores from Year 1 were used as covariatcs for the 
Grade 2-3-4 and Grade 4-5-6 samples. Metropolitan Heading 
Readiness Test scores were used as covariatcs for the Grade 
1-2 sample. Table 6 shows the mean post raw scores adjusted 
for unequal N and the effects of the covariatcs. Full 
statistical data arc available in Appendix G.) 
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Tabic 6 



Post Mean Math Scores for IP! and Control Children in the 
Grade 1-!:, Grade 2-3-4 an.! Grade -1-5-6 Longitudinal Samples' 



Sample Group 


ni, a:: post maip ramus 


Computa- 
tions 


Superior 
Group 


Concents 


Superior 
Groun 


An^J i- 
cations 


Superior 
Group 


Gr. U2 IPI 

Control 


21.23 
22.38 


i\onc 


18. 1 3 
18.42 


None 






Gr. 2-3-4 IPI 

Control 


14.01 
17.47 


Control* 


14.73 
15.36 


None 


11.55 
10.70 


None 


Gr. 4-5-6 IP] 

Control 


11.76 
17.1LS 


Control** 


11.65 
12.41 


*** 
Control 


14.47 
; 13.83 


Mono 



*p_<.0008 
**p <.0001 
***£<• 085 



Tabic 6 shows that the only significant differences 
in any of the three longitudinal samples were 311 favor of 
the Control schools. In both the Grade 2-3-4 and Grade - — 
4-5-6 samples, Control chi Idrcn j5CoSI«siKntficantly 'higher 
than IPI children on the Computations subtest, ^lit tiic Grade 
4-5-6 sample, Control children scored higher on the Concepts 
subtest to a decree that approached statistical significance. 

To show tiic math performance of the longitudinal 
samples in relation to the grade level norms, the average 
rav: scores of the IPI and Control children were transformed 
into grade equivalent scores. These arc presented for each 
year of program operation in Figure 5. 
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Figure 5 shows that at t!ie end of t!ic first year 
in the program the children in the Grade J-2 sample in 1P1 
schools were performing at grade level, but that at the end 
of two years, they had begun to fall behind the norm on 
the Concepts subtest. (The line representing the scores 
for the 1PJ children is not readily apparent in the graph 
showing nerforniancc on the Computations subtest, because 
>• exactly coincides with the norn line.) 

In the Grade 2-3-4 sample, children in the IPI 
schools fell progressively further behind the norms on the 
Computations and Applications subtests until by the end of 
three years in the program they were approximately one 
year below the norms. On the Computations subtest, the loss 
was greater than that demonstrated in Control schools. On 
the Concept? subtest, however, IPI children lost ground 
relative to the norms during the first two years but made 
substantial progress in closing the gap in the third year. 
By the end of three years in the pre ;ram they were only two 
months below the norm on the Concepts subtest. Control 
children, who had started at the same level as the IPI 
children, were meanwhile performing two months above the 
norm at the end of three years. In the Grade 4-5-6 sample, 
a cumulative performance deficit was evident throughout 
the three years of operation for both IPI and Control 
children. On the Computations subtest the deficit was 
much greater for IPI children than for Control children, 
with the IPI children performing two years below the norm 
after three years in the program. On the Concepts subtest, 
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IPI children began the program achieving, at the norm level, 
but fell a year behind the norrs after three years in the 
program to a level conpnrablc with that of Control children. 
On the Applications subtest the deficit was greater for 
Control children, bur the IPI children had still fallen 
approximately a year and a half below the norms at the end 
of throe years. 

Summary of Longitudinal Analysis 

Comparison of the math achievement results of 
children who had received the services of the IPI program 
continuously for two or three year periods with that of 
children enrolled in Control schools for the sane period 
showed that where significant differences occurred, they 
were in favor of Control children. In rost cases, children 
locciving IPI services for two or three years fell progres- 
sively further below the grade level norms over the course 
of time. 

It should be noted that the longitudinal samples 
at the end of Year 3 arc not exactly the same proups of 
children as the samples at the end of Year 2 because of 
pupil mobility. As mentioned earlier, with each succeeding 
year, the group of children in both 111 and Control schools 
who have been there since the program began gets smaller 
and more select. Restricting the longitudinal sample to 
geographically stable students nay introduce a bias, the 
extent and effect of which is not known. 



- 41 - 



Cross Section:!] Analysis of A chi evement 

a • Cross Sectional Da t a Co I lect ion 

The cross sectional analysis of .achievement attonpts 
to. answer the following type of question: After three years 
of operation of the IPI Prog/an, how are children in Grade 3 
performing as compared with the performance at Grade 3 when 
the prof ram began? Cross sectional grade level comparisons 
of this' sort were made using the results of the reading and 
arithmetic subtests of the Comprehensive Tests of Basic Skills 
(CTBS) obtained through the city-wide testing program. 
Mean scores on the subtests were obtained for the IPI 
schools, the Control schools and for all 30 Title 1 target 
schools. The test results from the 1069-70, the l<m)-71 
and the 1971-75 school years (Years 1, 2 and 3 of the IPI 
Program) were cxarrincd at Grades 3, 5 and 6. The use of 
these years and grades was determined by the city-vide 
testing .schedule. Baseline data arc not available for the 
years preceding the IPI Program because the CTBS was not 
used in the city-wide testing program until the first year 
of IPI ♦ 

b. Cross Sectional Data Analysis 

Figure 6 shows the average r.ath performance levels 
of children in Grades 3, 5 and 6 in the IPI schools, Control 
schools and the Title I target schools during each of the 
three years of operation of the IPI Program. Because test 
dates sometimes varied, the data arc presented in the form 
of grade equivalent months deviation from the norm, rather 
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than »rade equivalent scores. Due to the lack of statistical 
control or. these data, apparent differences in the perforrance 
level between groups at a given point in tine should not 
be interpreted too strictly. What is of interest is the pattern 
of performance chanp.es by each group across the three years. 

The figure shows that at Grade 3 the changes in 
math performance over the three years of the I PI Program 
were generally parallel for IPI and Control schools. Per- 
formance rose in the IPI schools from Year 1 to Year 2, but 
the same pattern was observed in the Control schools and the 
other Title I target schools. In Year 3 performance levels 
fell again in both IPI and Control schools, while it continued 
to rise or remained the same in thetTitlc 1 target schools. 
The pattern of performance changes was similar on all three 
math subtests. 

In Grade S, there was again little difference between 
the IPI schools, the Control schools and the other Title I 
target schools in the pattern of pcrfornance changes between 
Year 1 •and Year 2. In Year 3, iiowcvcr, the pcrfornance 
levels in the IPI schools dropped precipitously, although 
there was little change in the performance levels in the 
Control or other Title I target schools. The pattern of 
performance level changes was consistent across all three 
subtests ndmini stored. 

In Grade 6, the pattern of change in perforrance 
levels was similar among all three groups of schools on the 
Computations subtest. On the Concepts and Applications sub- 
tests however, there was some difference between the changes 
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thai occurred in the 11*1 schools and Die changes that 
occurred in the Control and other Title I schools. Between 
Year 1 and Year 2 the changes in performance were similar 
for the three groups, a slight rise in performance levels. 
In Year 3, however, performance levels dropped in the 
Control and other Title I schools, but continued to rise 
in the IP1 schools, In Year 3 performance in the IPI schools 
was four months higher than in Year 1 on both the Concepts 
and Applications subtests. In the Control and other Title 
I schools performance levels in Year 3 were generally the 
same or lower than in Year 1, especially on the Applications 
subtests. 

Summary of Cross Sectional Analysis 

In examining changes in nath performance at three 
grade levels across the three years of operation of the IPI 
Program, any such changes mist be interpreted in light of 
the changes occurring in other schools. In this cross 
sectional look at achievement, there was some evidence of 
inpact of the IPI Program at Grade 6: on two subtests 
a pattern of slowly improving pcrfornancc in the IPT schools 
took place in the face of generally declining performance in 
other schools. The pattern was by no means conclusive, 
however, and several norc years of data would be required 
before any definitive statements could be made. In Grade 
3 the changes in performance levels in IPI schools were 
generally paralleled by sinilar changes in the Control 
and other Title 1 target schools. At Grade 5 the per- 
formance changes' in the IPI schools compared unfavorably 



with the changes in the other schools. In general, the 
cross sectional analysis yielded little evidence that the 
IP1 Program has had a solid impact on achievement levels in 
the schools served by the program. 
V. CONCLUSIONS AM) RilCOMMI-KDATIONS 

A . l l21± u ,; i I ill °/ j} ° r li JLLS 

The results of the third year of operation of the 
I PI Program have shown that (as far as can be determined 
without a detailed analysis of specific diagnostic and pre- 
scription writing procedures) the I PI Program continues to be 
implemented according to pl*m. The question that remains is 
the effectiveness of the "retrain in raising the nath achievement 
levels of the children served. The prc-post analysis of 
progress through the sl.i 11 levels within the IPI continuum 
shoved that the children are moving along the continuum, but 
because of the lack of norms against which to compare their 
progress, it is difficult to determine whether they arc pro- 
gressing at a "normal" rate. The cvaluator rust, therefore, 
rely on information from standardized achievement tests to 
draw conclusions about the impact of the program on* math 
performance. 

The achievement data from the third year of operation 
showed that the children in 1P1 schools fared somewhat better 
in comparisons with children in Control schools than they did 
in previous years* The results, however, did not indicate a 
strong impact, except at Grade 6. Similarly, cross sectional 
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analyses of chanros in performance levels at riven grade * 
levels across t!ic three years of operation showed little 
evidence of ii-proYurccnV except at Trade 6. Lor.jutudj nal 
analyses of the- performance of only those children who had 
been in the program continuously for two or three years showed 
no evidence at all of superior pcrfornaiicc by J PI children. 

As discussed in the evaluation report for* "Year 2 of 
operation, there arc several possible reasons why a greater 
impact on achievement has not been evident. The first 
possibility is, of course, that the IPI approach simply docs 
not work any better than the traditional Methods of isath ■ 
instruction. Other possibilities exist, however. It is 
possible that the standard.ir.ed achievement results do not 
adequately measure what is taught in the IPI continuum, (a 
possibility tiiat raises questions as to what is taught in 
the 1PI continuuu). One way of investigating this thesis is 
to analyze the standard zee! tests iter.! by iters to detefriinc 
whether the questions reflect the skills and concepts covered 
in IPI • The tcajn of consultant teachers in Stephen Howe 
school was assigned this task, and their analysis of the 
arithmetic subtests of the Stanford Primary 1, Stanford Primary 
II, Stanford Intermediate I and Stanford' Intermediate II 
achievement tests showed that an average of 94% of the itens 
on these tests were covered in the IPI continuum. The range 
of correspondence between the Stanford tests and the 1P1 
continuum was fron to 100o for nine arithmetic subtests. 
It would appear that the standardised tests do measure the 
sane things that are taught in the IPI continuui.;. — 
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Il is possible, of course, vven though the standard? rot 
tests r.easure basictlly T'ic~ snre s!:iiis as arc tauj-ht in IPI, 
that tne lanpiaj'.c of the tests is sufficiently different fron 
the ]anr,uaf,c uscu in IP1 to inpair performance on the tests. 
Vor example, one question in the Stanford Intermediate 1 
Arithmetic Concepts Hakes use of the tcrrcs mil tiplj or, mil ti - 
plicand and ninue nd, but these terns arc not used at all in IPT. 
To investigate tnis potential problem, the consultant teachers 
in the 1PI schools constructed achicvcr.cnt tests directly fro;? 
jL- the 1PJ materials. Tests were constructed for Grades 5 and 5, 

1 ; and a'tCias were selected that, in the judgment of the 1PI staff, 

reflected the instructional activities in which all ov most of 
— ■ . the pupils had participated. Since the test items reflected 

what r:o*;t of the children had been taujiht in the IPI continuum, 
a nastcry level .performance was expected. It \;as anticipated 
that each iter: would be answered correctly by at least 75* 
of lite children. The results showed, however, that no norc 
than 36" of the items on any of these tests were correctly 
answered by 71>-c or norc of the children. In fact no r.orc than 
_ 57% of the items on any of the tests were correctly answered 

by norc than 60% of the children. Those, results arc not con- 
clusive, because these IPI tests are as yet in unrefined form, 
but they do surest that lack of evidence of improvement in 
math achievement nay not be due to •the use of standardized 
tests. 

Another possibility for explaining the apparent lack 
of progress in raising achievement levels is that the IPI 
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systeu teaches the ski 1 hi Hint the Cleveland Schools v.ants 

taught, but that it teaches then in a different sequence from 
that nor; -ally folloi.c.i. That if, by the lisr.o !ic leaves 
elementary school, a child in 1P1 nay have covered the same 
concepts and skills as a child not in IPI (hopefully with 
greater rastery) , but at any given point in time during their . 
elementary school careers, they ray be working on entirely 
different skills. If this is the case, the only valid asfess- 
went of 1 PI possible is a longitudinal study in which achieve- 
, went levels of IPI and Control children arc compared only after 
the IPI children Have spent all or most of their elcnentary 
school years working in the 1 PI systen. One problem with this 
approach, however, is that the percentage of children remaining 
in the schools that long is relatively skmI], Less than one third 
of the children present in the IPI schools during Year 1 were 
still in the schools by the end of Year 3. "Therefore, the 
whole question of whether the IPI Program night have an inpact 
on a evenent if children participated for their entire 
elementary school tenure way be acadenic. 

Finally, despite the indications that the progrnn 
guidelines have been faithfully followed by the IPI staff , over 
the course of three years it is possible that deviations from 
the precise testing and prescription criteria established by 
the program's designers have crept into the operation. If so, 

—these deviations i:;ay be reflected in lack of improvement in 

math achievement. It was recounended in the evaluation report 
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for Year 2 that aijMhcr tva Iumi ion of oper-t io:ss accuracy he 
conducted by Research for heifer School. 1 ;, hut that organisation 
has reported xhat l.icy no longer perform t hat scrvicv;. Seif- 
r:o!!iiori»; a instruments have bee:*, developed, however, and their 
use in the Cleveland IP1 schools nay yield valuable inforra;ion 
as to the accuracy with which the 1 PI procedures are being 
fol lowed. 

In the long run, the decision about the future of the 
11*1 Prop.ran will have to balance prop.ran costs against program 
benefits. There is sone evidence that achievement levels in 
math in the IPI schools nay be inwoviiiR slightly at certain 
tirade levels. i>ut is is ;ilso clear that over the course o* 7 
liirce years the cost of the prop.ran hrs s risen 7-1' to S210 per 
pupil. The question becor.es whether the present or future 
pains in achievement are f^reat enough to justify the expense*, 
o)' whether sorts efficient and less costly avenues to t!ie sare 
ends can be found. 

At any rate, the results of field tests of experimental 

proprans in the schools rust be interpreted in lifht of the 

nurber of influences over which the evaluator has no control; 

experimental treatnent populations change, control populations 

change, samples nay become biaseci th/oup.h non-randon attrition. 

All of these factors way cause changes J_n performance that have 

nothing to do with the caper jnental treatment under invest jjta* 
« 

tion. livaluation j::iu;t, therefore, tal:c a lonp and cautious view 
results and data must be examined over a poricd^of several years 
before administrators can state v;ith sone decree of confidence 
that ;! j'.ivcn prof.rai.i is or is not effective. 
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1. Jt is recur r.ended that tac 1!M Pivfr&n he continued in 
the sp.no tv:o sih'^J ?• i;: v.iich it prt-serl* !y oyeratc:-" f*»r 
the next one or t*:o years until suffici out data arc 
available for an accurate appraisal of lonj* tern pro?jvM 
effects. 

2. It is rceo: tended that a decision on expansion of the 

IP] I'royrai.: lo other schools be deferred instil the results 
of a lonp. tcsvi evaluation are nvailaMe. 

3. It is recommended that, the devoloprent and refinement of 
achievement tests constructed frop the IP! material r. 
proceed. 

4. It ir ; roc^'^c;::?^ 1 t ! :"i* rrlf-nonitorinp procedures he 
instituted in each Ji 1 ! school, usin« the Snsrruncnts 
developed for this purpose by RBS. 
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APPhNMX A 
Descript j vc Data on Pro j oct Sclioo 1 s 
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appknoix c: 

Math Achievement - IPI vs. Control 



Adjusted Mean Pre and Post Math Achievement Test -Scores 
for J PI ai:d Control Schools at Grade 2 with 
Results of Multivariate Analysis of Covarianec 
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KOTli: Analysis v:as performed on post test secures with pre test and PLR 
scores as covariates. Covariates shown arc adjusted for unequal 
N. Post test scores shown are adjusted for unequal N and covariate 
effects. Stanford Primary I used as- pre test and Stanford' Primary 
II used as post test. 



The Stanford Prinary I ];jvcn as the pre test in Grade 2 has only 
one arithmetic subtest. The scores fron this subtest were used 
as the prc-scores for the Computations and Concents subtests in 
the Prinary II, which was .fljven as the post test. 
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APi'i.NHlX C Continued 



Math Aciti evenent - J p l vs. Control 



Adjusted -lean Pre and Post Math Aciiicvcncnt Test Scores for 
1EL and Control Schools at Hrade «1 u-ith Results 
of Multivariate Analysis of Covariance 
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MOT!:: Analysis was performed on post; test scores with pre test and PLR scores 
as covariatcs. Covariatcs shown are adjusted for unequal \ T . Post test 
scores shown arc adjusted for unequal N and covariatc effects. Alternate 
fornss of Stanford Intermediate 1 used as pre and post tests. 



APPhNlil X C Continued 



Math Achicv vno; U - I PI vs . Control 

Adjusted Mean Pre and Post Math Achievement Test Scores for 
I PI and Control Schools a<" Grade 6 with Results 
of Multivariate Analysis of Covariancc 
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NOT);: Analysis was performed on post test scores with pre test and PI,R scores 
as covariatcs. Covariales shown are adjusted for unequal N. Post test 
scores shown are adjusted for unequal >! and covariate effects. 
Alternate forms of Stanford Intermediate II used as pre and post tests. 
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A c h i cvcr.-nt - Boy s vs. Girl s 

Adjusted .Mean Pro ana Post Ai.iievcncnt Test Scores for hoys and Girls at 
Grade 2 with KesuJts of M.ul tivari ate Analysis of Covariancc 



Subtest 


Mean Pre Scores 
(Covnr i atos) 


Mean 
Post Scores 


Least 
Square 


Uni vari ate 






hoy s 


Girls 


Boys 


'"•iris 


list irate 


V 


df 


• . , • -| 

An timet j c 


34.. S6 


56.58 












Ari timet i c 
Computations* 






22.47 


20.56 


1.92 


3.92** 


1 f, 2HJ1 


Arithmetic 
Concepts 






18.01 


15.19 


2.83 

i 


19. IS* 


1 f, 209 


PLR 


10B.3 


in. A 




Multivariate l ; 


5= in.ss* 


2 208 
i ..... - — 



*£< .0001 

**£< .05 



N0T1": Analysis was performed on post test scores with pro test scores and 
PLR scores as covariatcs. Covariatcs shown are adjusted for unequal 
N. Post test scores shown are adjusted for unequal N and covariate 
effects. Stanford Prinary I used as pre test anJ Stanford Prinary 
II used as post test. 

a Thc Stanford Prinary I Achievement Test pven as the pre test in 
Grade 2 has only one arithmetic subtest. The scores from this sub- 
test were used as the pre scorer? for the Computations and Concepts 
subtests in the Stanford Primary II which was given as the post test. 



APPHNUIX D Continued 



Ac hiever c n t - hoy s vs. Ri rl s 

Adjusted Mcr.ii Pre and Po'*»t AcJi \ cvci*ent Test Scores Tor Boys and Girls at 
Grade 4 with Results of Multivariate Analysis of f.'ovarMttcc 



Subtest 


Mean I'rc Scores 
fCovnrj Mr rO 


:?c 


Score 


Least 
Square 


Uni vari ate 






!»oys 


(li ris 




''.i ri : ; 




P 


df 


Arithmetic 
Computations 


8.36 


10.25 


11.68 


14.80 


- .11 


.21 


] .; 206 


Aritht ictic 
Concepts 


9. IS 


9. OA 


14.20 


IS. 77 


.51 


.69 


1 f, 206 


Arithmetic 
Applications 


R.r> f ; 


8.58 


] 1 . 25 


10.36 


.88 


3.63 


1 f, 206 


PLU 


04.68 


PP. 57 




Multivariate Y 




7, f, 204 



*p <.05 



NOTH: Analysis was performed on post test scores i:ith pre test and PLU 

scores as covariates. Covarintes shown arc adjusted for unequal N. 
Post test scores .shown are adjusted for unequal N and covariatc 
effects. 
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APPINMX P Continued 
Achieverent - hoys vs. Hi ?•] s 



Adjust od l-.L'in Pre umi Post Aciiievor'/'nt Test Score-;-, for Boys and Girls at 
Grade 6 with Results of Multivariate Analysis of Covariancc 



Subtest 


Mean l> 
fCnv 


re .Scores 
nr? at cs) 


Post 


•an 

Pcorcs 


Least 
Square 


Univan ate 






Boys 


Girls 


iJoys 


< : i rl p 




V 




Arithmetic 
Computations 


10.01 




n.32 


14.53 


1 5.65 


- 1.32 


5.37*** 


1 f, 323 


Ari Ihisctic 
Concepts 


' 9.69 




9.08 


12.10 


J 1 .47 




5.91** 


1 5 323 


Arithmetic 
Appl i cat ions 


| 3.1.69 




12.00 


14.49 


13.97 




1.66 

• 


1 f, 323 


PLR 


95.63 




98.82 




?iul 1 i vari;it<; 1 : 


r. 5. SO* * 


3 f, 321 



*p< .POOS 
f *P< .016 
***£< .022 



NOT!;: Analysis was performed on post test scores with pre test and PLR 
scores as covariates. Covariatos shown arc adjusted for unequal 
N. Post test scores shown are adjusted for unequal N and 
covariate effects. 
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APPENDIX I* 



Achi cvencnt - li owc vs, Parkwo od 

Adjusted Mean Pre and Post Achievencnt Test Scores for Stephen Howe and 
Parkwood Schools at Grade 2 with Results 





of 


Multivariate AnaJy: 


;i s of Gov 


ariancc 








Mean Pre Sec res 
(Covnriafcs) 


Mean 
Post Scores 


Least 
Suuarc 


Uni - 
vari ate 




Subtest 


jiowr 


Parki.voc! 


howc 


Parkv.ood 


list i nave 


V 


df 


Arithmetic 0 


3<>.il 


3.1.25 












Arithmetic 
Computations 






20.22 


22. 22 


- 2.00 


J. 94 


1 f, 200 


Ari timet ic 
Concept s 






34.23 


18.36 


- 4.13 




l r, .?np 


PI.R 


108.1) 


101.7 




Multivariate V 




2 f, 2ns 



A |^< .0001 
**p < .0002 



NO'l'ly .(Analysis *was performed on post test scores with pre test scores and 
. VIM scores as covariates. Covariates shown are adjusted for unequal 
N. Post test scores shown are adjusted u>r unequal \' ;uid covariatc 
effects. Stanford Prirary I used as pre test and Stanford Prjnary 
II used as post test. 

a Thc Stanford Prinary 1 Achievement Test piven as the pre test in 
Grade 2 iias only one arithmetic subtest. The scores from this 
subtest were used as the pre scores for the Computations and 
Concepts subtests in the Stanford Primary IJ which was piven as 
the post test. 
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Al'P!.\i*! X i; i'om j i *uo.! 



Achievement ~'lkn:c vs. Parkwood 



Adjusted Mean Pre and Post Achieves .cut "j>st Scores for Stephen Hove and 
P;jrk\:ood Schools at Grade 4 with Results 
of Multivariate- Analysis of Covarianoe 























Mean P 


re Scores 


Mean 


Least 


Uni - 








(("ova 


ri ales ) 


post 


Scores 


Souarc 


varj ate 






■ Subtest 


ilovri- 1 


Pari i ;oo»i 




i ari.'.ooci 


i" st i pale 


V 




df 


Arithneti c 


















Computations 


9.07 


8. Of) 


14.5?; 


11.10 


5.48 


10.J3* 


1 


Fi 206 


Arithmetic 












*-** 






Concents 


8.6l» 


8.84 


14.25 


12.64 


1.61 


3.56 


1 


f, 206 


An thr.eti c 


















Applications 




7.52 


11.27 


J}. 18 


no 
• • ■ 


.27 


1 


f, 206 


1M/R 


08.9.'! 


P9.13 




Multivariate F 


= 4.02*-* 


1 3 f, 204 

T 



*P <.O02 
**£ < . 00S> 
***p <.06J 



NOT!:: Analysis i;as performed on post test scores with p^c test and PI.R 
scores as covariotcs. Covariatcs s!iov;n are adjusted for unequal 
N. Post lest scores s:iown arc adjusted for unequal M and covariatc 
effects. 
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AP! YS'HX * ; Continued 



Ach*5 ovc-'^'m - !!o;:e vs, Par! v.v-d 



Adjusted .Mean Pre and Post Aoievu.ent T'-st Scores for Stephen Howe and 
ParJ.i.^od Scheols at Crad: 6 vii!s U-«:ulis 
of Mull 3 variatc Analysis of Covaviamx- 



■• — — — — - — - — - 


Mean P) c; Scores 
(Covariatos) 


Post Scores 


•" — ■ 
Loasv 
Square 


vari nto 




Suhlcsi 


! >o;:e 


Pan \;ikk! 


ilOi.'f 


i';iv : '..nod 


i-M i:n« c 






AritfiMctic 
Computations 


11.88 


8.02 


1.4. 


12.32 


2.35 ; 


KUIS* 


i r, 


Ari timet ic 
Conccptr. 


9.93 


8.07 


12.27 


12.27 


; 

-.001 


. 003 


1 5 323 


Ari£hrsctic 
Appl i cations 


13.03 


10.93 


14.91 


15.58 


-.67 


!.16 


1 f, 323 


PLR 


100.1 


92.55 




'lultiv.-rjjitfi ! : 
j . i 


= 4.56** 


3 r. 521 



*0 < .002 



N0TK: Analysis v.as norforncd on post test, scores with pre tost and PL 1 ? 
scores as covariat es. Covariatcs shown are adjusted for unequal 
N. Post test scores shown arc adjusted for unequal N and covariat c 
effects. 



APPi.NiJlX !■" 



Fori? one 1 Level o f Stanford Achicvcpcnt Yer.ts Administered In 
Year 1, Year 1 am! Year 3 to Pupils in tLe !.om;i riuii naJ Snnplc*. 







Sui ;>3 f Si r,<- 


Level and Torn of o:>t 


(Jradc Pro *r-: , > e;;r 


i i'l (!o:siml 


Post - Primary 1, u 

Pre - Prinary I , X 
Post - Prinary J I , K 


1 2 

2 3 


31 655 


Pre - Prinary I , U 
Post - Priwary II, W 

Pre - Prinary II, X 
Post - Priifcirv 11, 

Pre - Interned} ate I, X 
Post - Interned i ate I, K 


2 1 

3 2 
<\ 3 


55 62 


Pre - Inter:- ediate I , W 
Post - Intcrocdiate 1, X 

Pre ~ Intermediate IT, X 
Post - Interrelate 11 , L r 

Pre - Intermediate II, X 
Post - Intermediate II, IV 


4 1 

5 2 

6 3 


72 127 
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APPLNMX C 

Achi^ ver^^t - " : [ vs. Co:>t iyi l o w \Vi -ji_na l 

Adjust oil Mean !Vrt Aciticvcc.it Test Scores for IP] and Control Children 
5 11 i he Longitudinal Sa;;.Us with Results of 
Multivariate Analysis of ;:ov:iri:.iscc 

Crnde 1 - 2 



SllMcSt 


Mean J 
Post Scorns 


Ua:.l j 
Soi;r.re 1 


Univariate 








in 


Control I 




F 


c!f 




AHthnotic 
Co:, put at ions 


21.23- 


22.38 






1 and 


PO 


Arithret to 
Concents 


"JS.15 


IK. \ 




- . 29 


. 003 


1 ai.d 


W 


Metropolitan 
Iie;uMr.«: Peadi- 
noss lest 
(Covariatc) 


72. Of) 


















:!uln*variate !■ 


* .45 


| 2 and 




NOT! I: Analysis 


\;as jicrforrcd 


on 


Grade p^st test scores 


t:i tii 





Metnmol ilan Heading P.cauincss Test score as the covariatc. 
Co va Hale shown is adjusted for unequal N. Post tost scores 
shown are adjusted for unequal N and the effect of the covariatc. 
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APPI..\i)IX (J Cent 1 j.ucci 

Aimi cv< i .ont - 'li. s vs. Control Lew i tudi n: 1 Pat a 

Adjusted Mean Pre and Post Ai.ii cvorcnt I est Scores for JPI and Control 
Children in tiic Lonri X udj iv 1 Srn.ples vith Results o r 
Multivariate Analysis of Covariance 



Crado :>-:<■<! 



Subtest 


Mean # Pro Torres 
(Covariatos) 


~Mean 
Post Scorer 


Least 
F.,ua)v 


Unj- 
variat e 






JIM 


Control 


I'M 


C( r.lro) 


i.stirato 


F 


df 


Ari trine tic a 


SO. 00 














Ari t !iret i c 
Cor.putation 






14.01 


17.47 


- .V17 


i O on* > 

J . 4.**. 


1 i\ 87 


Ari tiinetic 
Concept s 






14.73 


1 j . 36 


- .03 




1 f, 87 


Ari t arctic 
Appl icat ions 






11 .:>."> 


10//0 


.«5 


1 


1 f, S7 


WAX 


101.0 


100.1 




Mult: 


ivarialc ! ; 


r 7.'!?* 


3 r, s. r i 


"p<.0002 
**£< .0008 

















N0TI:: Analysis was perfornod on Grade 1 post test scores with fir ad e 2 pre test 
and PLR scores as covariatos. Covariatos shown are adjusted for unequal 
N. Post test scores shewn are adjusted for unequal N and covariate 
effects . 



a Thc Stanford Primary 1 Achievement Test Riven as the pro test in Grade 
} ' 7 lias only one arithmetic subtest. The scores from this subtest were 

used as the pre scores for t!ie Conputatious, Concepts and Applications 
subtests in the Stanford 1 nternediat c I which was piven as the post- 
test. 



~ 67 - 



AIMM-XDIX C Continucc! 



Acui eve rvnt - M LS vs . Cent rol L^i^ i t udi \u >. \ l>a t a 

Adjusted Mean Pre arid Post Achicvtnent Test Scores for and (font rol 
Children in t : i c- Loup tudi v.:\] S'anplos wit:i Results of 
Mu) t ivar j ate Analysis of Covariancc 

Grade 4-5-6 



Subtest 


Mean Pre Shoves 
(Covariates) 


Mean 
Post Scores 


Least 
Square 


Uni- 
variate 






)!'] 


Control 


1 PI 


Control 


1 st inatc 


! : 


df 


Ari thnot ic 
Computations 


in. is 




11.70 


17,98 


- 6,22 


87.39* 


l r, 187 


Ari i .met i c 
Concepts 


9.52 


9.61 


J 1 . 5() 


12.41 


- .76 




1 5 187 


AritMin t ie 
Appl i cations 


7 . 87 


(>.!>:$ 


14.47 


13.88 


.59 


.51 


1 187 


Pi.R 


99.24 


92. Si 5 




Mu] tivariatc i : • 


- 34.30* 


3 f, 135 



*p < .000] 
**p < .032 



N'OTl:: Analysis uas porfornoii on Crade 6 post test scores with Tirade 4 pre 
test and PLR scores as covariatcs. Covariatcs shown ,are-Vidjust ed 
for unequal N. Post test scores shown arc adjusted for unequal N 
and covariatc effects. 
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